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TRACK  INITIALIZATION  IN  THE 
MULTIPLE-OBJECT  TRACKING  PROBLEM 


Karel  ZIKAN* 


Department  of  Operations  Research 
STANFORD  UNIVERSITY 
Stanford,  CA  94305 


Abstract 

The  multiple-object  tracking  problem  involves  extraction  of  the  trajectories  of  n  moving  points  from 
(three)  successive  motion  picture  frames.  In  the  track  initialization  part  of  the  problem  no  previous 
history  of  track  evolution  is  given. 


A  definition  of  a  “three-point  metric”  functional  (analogous  to  the  classical  definition  of  distance) 
is  put  forward.  For  the  best  estimate  of  the  trajectories,  we  partition  the  points  from  the  frames 
into  n  triplets  (based  on  the  three  successive  frames)  so  that  the  average  three-point  “ distance ”  is 
minimized.  The  physical  intuition  behind  this  approach  is  discussed  and  several  equivalent  math¬ 
ematical  programming  formulations  are  given.  A  practical  method  proposed  for  solving  of  the 
problem  is  based  on  a  Lagrangean  relaxation  technique,  and,  to  a  lesser  degree,  on  the  “pruning"  of 
the  tree  cf  “subpartitions”.  On  the  basis  of  empirical  evidence  and  experience  from  related  work, 
«e  conjecture  that,  on  average,  0(n3)  arithmetic  operations  are  needed  to  obtain  a  solution.  The 
problem  of  missing  and  spurious  points  in  the  images  is  also  briefly  discussed.  A  short  summary 
and  examples  from  simulated  data  experiments  a  re  given. 
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0.  Introduction 


In  this  paper  we  address  the  multiple  object  tracking  problem.  We  are  given  three  consecutive  frames 
(one  can  think  of  a  movie)  of  the  same  scene.  The  frames  are  taken  in  regular  fraction-of-a-second 
intervals  and  contain  n  moving  points.-^The  track  initialization  problem  is  to  estimate  the  n  most 
probable  trajectories  that  may  give  rise  to  the  given  set  off frames,,  It  is  called  track  initialization 
problem  because  no  /‘history’"'  of  the  previous  tracks’  evolution  is  given/  The  track  initialization 
problem  is  a  harder  problem  than  the  also  important  track  extension  problem,  where  in  loose  terms 
we  need  to  extend  n  existing  trajectories  so  that  they  would  incorporate  the  set  points  of  the  latest 
frame.  In  general,  both  of  these  subproblems  of  the  multiple-object  tracking  problem  arise  naturally 
in  the  aerospace  and  defense  industries.  For  instance,  see  Arnold,  bar-shalom,  and  Mt/ccr  [2], 

The  “three-point  metric”  approach  to  the  track  initialization  problem  came  up  as  an  application  of 
algebraic  metric  theory,  zikan  [14],  Chapter  12  of  [14]  contains  the  information  presented  here.  The 
rest  of  [14]  contains  supporting  material  and  applications  of  the  theory  to  related  computer  vision 
nroblems  (e.g.,  detection  ot' missing  and  spurious  points).  However,  the  research  presented  here  can 
be  understood  without  studying  it  in  the  broader  context  of  the  algebraic  metric  theory. 

In  Section  1  we  introduce  the  notion  of  “distance”  between  three  points  simultaneously.  We  provide 
a  definition  and  some  examples. 

In  Section  2  we  begin  the  discussion  of  the  track  initialization  problem.  We  provide  the  motivation, 
discuss  the  goals,  and  identify  the  difficulties.  Here  also  the  track  initialization  and  track  extension 
problems  are  contrasted.  The  principle  of  momentum  preservation  provides  the  key  physical  insight 
and  motivates  the  model.  We  choose  one  of  the  three-point  “metrics”  to  serve  as  a  “penalty  function” 
for  the  estimated  trajectories.  The  n  (disjoint)  trajectories  that  together  give  the  least  total  penalty 
become  our  estimate. 

In  Section  3  we  propose  a  mathematical  programming  model,  and  give  an  algorithm  to  solve  it.  The 
proposed  solution  method  has  two  stages.  During  the  first  stage,  a  Lagrangean  relaxation  technique 
is  used  to  obtain  a  stationary  point.  Although  the  point  need  not  be  the  global  optimum,  it  usually 
will  be.  To  insure  global  optimality  as  well  as  to  find  good  alternative  solutions,  a  branch  and  bound 
tree  pruning  algorithm  with  the  stationary  point  as  the  initial  incumbent  is  proposed.  The  overall 
average  case  “running  time”  seems  to  be  about  0(n3). 

The  computational  complexity  issues  are  discussed  in  Section  4.  The  discussion  focuses  mostly  on 
the  empirical  average  case  running  time  requirements. 

Section  5  contains  a  description  of  a  simulated  track  initialization  experiment.  We  briefly  discuss  the 
problem  of  missing  and  spurious  points.  The  method  developed  in  [14,  Chapter  10],  is  used  here  as  a 
subroutine  to  “clear”  the  images  of  spurious  points.  Examples  and  figures  of  the  track  initialization 
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problem  in  various  stages  of  computation  are  presented  in  the  remainder  of  this  section. 

1.  Three-Point  “Distances” 

Distances  are  measures  of  closeness  or  similarity.  Given  a  space  X,  a  metric  d  can  be  viewed  as  a 
functional  that  maps  pairs  of  points  in  X  into  real  numbers,  d  :  X  x  X  — »  R,  and  that  reflects  the 
“degree  of  association”  of  each  pair  of  points.  This  interpretation  of  metrics  leads  inevitably  to  the 
idea  of  “distances”  between  three  (or  more)  points.  We  want  to  define  a  three-point  “metric”  V  as 
a  functional  that  maps  each  three  points  into  “reals”,  D  :  X3  — *  R,  and  that  captures  the  degree  of 
association  for  all  triplets  of  points. 

Definition:  A  functional  T>  is  a  three-point  (3-p)  metric  on  the  space  X  if  it  satisfies  the  following 
properties  for  all  ordered  triplets  of  points  x,y,z  €  X: 

PI:  T>(x,  y,  z)  >  0, 

P2:  V(x,y,z)  =  V(z,y,x), 

P3:  V(x,  x,  x)  =  0, 

P4:  inf„,€;r  V(x,  w ,  y)  +  inf„,€x  D(y,  *)  >  inf«,6x  T>(x,  w,  z). 

These  axioms  are  aesthetically  pleasing  for  the  way  they  correspond  to  the  normal  (two-point) 
axioms.  They  are  also  useful  as  we  will  see  later  in  the  chapter.  The  definition  generalizes  to  more 
than  three  points  in  a  transparent,  nested,  manner.  Here  are  some  examples  of  3-p  metrics. 

Example  1:  Perimeter  3-p  metric.  Given  a  normed  space  {X,  ||  •  ||),  we  define  V(x,y,z)  — 
II*  —  y||  +  lly  —  *11  +  II*  —  *||,  the  perimeter  of  the  triangle  A(x,y,  2).  Note  that  P1-P3  are  clearly 
satisfied.  If  y  is  a  convex  combination  of  x  and  z  then  infy6x  V{x,  y,  z)  =  2||x  —  zjl  is  achieved.  Thus 
the  triangle  inequality,  P4,  holds  too.  Note  that  the  axioms  are  also  satisfied  if  the  \\x  —  z||  term  is 
weighted.  If,  instead  of  a  normed  space,  we  are  given  an  arbitrary  metric  space  {X,d},  the  triangle 
perimeter  still  defines  a  3-p  metric.  The  infimum  inf„ex  V(x,  y,  z)  —  2 d(x,z)  is  still  achieved  for 

y  =  *• 


Example  2:  Centroidal  3-p  metric.  Consider  the  euclidean  space  En .  Let  c  be  the  centroid  of 
x,  y,  and  z  and  define  V(x,y,z)  =  ||x  —  c||-f  ||y  —  cj|  +  ||x  —  c||,  the  sum  of  the  distances  to  the  center 
of  the  triangle  A (x,y,  z).  Again,  P1-P3  are  clearly  satisfied.  Note,  that  infye*»  ||c  —  y||  =  0  is 
achieved  when  y  =  c,  and  infe€ft»  ||x  -  c||  +  ||c  —  *||  =  ||x  -  z||  is  achieved  when  c  =  ^1^.  Therefore, 
if  y  =  c  =  the  infest*  P(x,y,  2)  =  ||x  -  z|)  is  achieved.  We  can  now  see  that  V  satisfies  P4. 

Example  3:  Inertial  3-p  metric.  For  the  euclidean  space  En  and  scalar  a  >  0,  we  define 

P0(*,y,  *)  =  II*  -  y||  +  II*  -  y||  +  aW—^~  ~  yll-  (i  —  i) 
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Figure  1.1  shows  the  line  segments  lengths  of  which  are  the  components  of  the  functional  V(x,  y,  z). 
The  functioned  is  a  3-p  metric,  as  it  is  trivial  to  verify  the  axioms  P1-P3,  and  the  “triangle  inequal¬ 
ity”,  P4,  follow  easily  after  noticing  that  infygH«  /D(x,ylz)  =  j|a;  —  z||  is  achieved  when  y  =  s^£-. 
This  metric  is  used  in  our  approach  to  the  track  initialization  problem. 


Figure  1.1  Components  of  the  inertial  3-p  metric 

2.  Track  Initialization  Problem 

The  essence  of  the  track  initialization  problem  is  to  “match”  triplets  of  points  —  one  from  each 
image  frame  —  so  that  the  collection  of  trajectories  producing  these  triplets  best  satisfies  our  notion 
of  reasonableness. 

From  the  practical  standpoint,  track  initialization  is  a  difficult  problem  only  because  of  technological 
limitations.  If  we  could  take  the  frames  in  a  fast  enough  sequence,  effectively  making  a  “movie”, 
then  the  extraction  of  the  trajectories  would  be  simple.  Unfortunately,  for  some  applications,  the 
camera  speeds  are  inadequate  and  the  distance  between  trajectories  may  be  less  than  the  distance 
traveled  by  points  between  the  frames.  Some  trajectories  may  also  cross  in  the  time  before  the 
second  frame  is  taken. 

The  principal  idea  of  the  method  proposed  to  solve  the  track  initialization  problem  is  simple.  Physical 
objects  do  not  move  with  arbitrary  speeds,  and  they  obey  the  inertial  laws.  The  velocity  should 
change  only  continuously  and  gradually.  Good  trajectories  must  satisfy  these  requirements.  If  p,  q, 
and  r  are  “successive”  points  belonging  to  the  same  trajectory,  it  is  natural  to  require  that  (in  the 
terms  of  the  usual  euclidean  distance): 

t.  the  points  be  close  to  each  other; 

(2-1) 

««.  the  middle  point  q  be  close  to  q'  =  the  point  halfway  between  p  and  r. 

We  build  our  model  around  these  requirements. 
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Recall  the  3-p  metric  of  Example  1.3.  With  each  triplet  of  points  we  associate  as  a  penalty  the 
function  (1-1)  with  the  default  value  of  a  specified  as  a  =  4.  We  need  to  find  a  set  of  n  (disjoint) 
triplets  of  points  which  minimizes  the  total  penalty.  There  are  exactly  (n!)2  feasible  sets  available. 
Therefore  an  explicit  enumeration  of  all  possibilities  is  out  of  the  question  for  all  but  the  smallest 
problems.  (With  ten  objects  there  are  more  than  1013  feasible  sets  of  triplets.) 

The  rationale  for  using  the  3-p  metric  (1-1)  as  the  penalty  comes  from  the  assumption  that  the 
frames  were  taken  at  the  (technologically)  fastest  possible  rate.  Large  displacements  as  well  as 
large  changes  in  velocity  are  thus  unacceptable.  The  functional  T>a  “penalizes”  high  speed  and 
acceleration  in  trajectories. 

Let  us  (generically)  refer  to  the  points  of  the  first,  second,  and  third  frame  by  p',  q1 ,  and  r* 
respectively.  We  suppress  the  parameter  a  and  use  the  shorthand  notation,  =  Va(p',q’  ,rk). 
The  first  two  terms  of  the  values  ||p‘  —  q>  )j  and  Hr4  —  ||,  represent  the  penalty  for  displacement. 

The  last  term,  <*||Mp - ^  H  is  the  penalty  for  inertia  change. 

The  value  of  a  allows  a  relative  weighting  of  the  respective  parts  of  the  penalty  function.  The  choice 
of  a  is  slightly  arbitrary.  In  fact,  the  weight  parameters  provide  a  user  with  a  simple  means  of 
control.  By  changing  a,  we  alter  the  method’s  degree  of  preference  for  linear  and  monotone  (no 
acceleration)  motion.  The  default  weight,  a  =  4,  is  motivated  by  the  geometry  of  the  problem. f  In 
(stylized)  experiments  on  trajectories  generated  by  a  set  of  parametric  curves  (Section  5),  only  the 
default  weight  was  used.  The  correct  trajectories  were  selected  in  all  test  problem  instances. 


f  Consider  the  collinear  triplet  of  points,  p,q,  and  r,  produced  by  an  accelerating  point.  Since  we 
assume  that  the  frames  are  taken  in  regular  1-second  intervals  (let  t  =  1),  v  —  ^  (j|q  —  p||  +  ||r  —  q|j) 
is  the  estimate  of  the  average  velocity,  and  a  =  |jr  —  q||  —  \\q  -  pjj  is  the  estimate  of  the  acceleration. 
Because  the  points  are  collinear  a  simple  computation  verifies  that  P^(p,  q,r)  =  2(t>  -Ho). 
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3.  Formulations  and  Computational  Methods 


Integer  linear  programming  formulation.  The  integer  linear  program, 

n  n  n 

i=1 ;=1 k=l 
n 

subject  to  ^2Xi^k  =  f°r  Paif8  jkt 

*=i 

n 

^2Xiik  ~  *>  f°r  all  Pairs 
j=i 


(3-1) 


—  1,  for  £ill  pairs  ij, 

*= i 

and  xijt  €  {0, 1), 

is  the  mathematical  description  of  the  formulation  described  above.  It  is  a  generalization  of  the 
assignment  problem,  papadimitriou  and  steiglitz  [13].  The  variable  Xy *  =  1  if  and  only  if  the 
triplet  of  points  p*  and  rk  is  grouped  together.  Because  the  constraint  matrix  of  (3-1)  is  not 
unimodular  we  can  not  relax  the  integrality  constraint.  The  optimal  solution  set  to  the  linear 
programming  relaxation  need  not  contain  any  integral  vertices,  garfinkel  and  nemhauser  [8].  This 
is  an  unfortunate  computational  complication  in  comparison  to  the  assignment  problem. 

Diliuear  programming  formulation.  We  can  avoid  the  integrality  constraints  if  we  formulate 
the  track  initialization  problem  as  an  equivalent  bilinear  programming,  konno  [11],  problem: 


min^Dij*  •  x0  •  yjk 

ijk 

n 

subject  to  ^  x,j  =  1,  for  all  », 
i=i 

n 

=  1,  for  all  j, 


(3-2) 


and  to  =  1,  for  all  Jfc, 

j=i 

n 

=  1,  for  all  j, 

k= 1 

y  >  0, 

where  the  indices  range  over  the  integers  {1,2,. ..  ,n}.  Formulations  (3-1)  and  (3-2)  are  equivalent 
in  the  sense  that  from  a  solution  of  (3-1),  we  can  construct  a  cornerpoint  solution  of  (3-2)  and  vice 
versa.  Furthermore,  if  we  are  given  a  solution  of  (3-2)  which  is  not  a  cornerpoint  of  the  feasible 
region,  then  it  is  a  simple  matter  ( 0(n 3)  worst  case  complexity)  to  obtain  a  cornerpoint  solution  of 
(3-2)  as  well.  These  aspects  of  multilinear  programming  are  discussed  in  Section  8.3  of  zikan  [14]. 
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Trilinear  programming  formulation.  There  is  a  certain  lack  of  symmetry  inherent  in  the  for¬ 
mulation  (3-2).  This  can  be  rectified  by  formulating  the  problem  as  (yet  another)  equivalent  math¬ 
ematical  programming  problem.  It  is  the  following  trilinear  program 

n 

-EE  T>ijk  x.i  ■  y.i  ■  z.k 

,9‘  i  =  l  ijk 

n  n 

subject  to  *»«  =  1*  f°r  a^  *•  52  r*‘  =  ®3r  s'  x  — 

'V  *V  (3-3) 

52  y>i  = for  a11  52 =  for  a11  s>  y  £  °> 

<=i  >=i 

n  n 

52 z,k  =  ^°r  a^  52 z,i  =  ^  ^or  s< 2  — 

j=l  k= 1 

where,  again,  the  indices  range  over  the  integers  {1, 2, . . . ,  n). 

Of  all  these  formulations  of  the  track  initialization  problem,  the  trilinear  mathematical  programming 
problem  (3-3)  makes  the  best  computational  use  of  the  special  structure.  The  feasible  region  is  a 
direct  sum  of  the  three  orthogonal  polyhedra.  The  only  coupling  of  the  variables  x,y,  and  r  is 
via  the  inertial  term  of  the  objective  function.  It  prohibits  a  complete  decomposition  into  three 
independent  assignment  problems. 

Lagrangean  cyclic  block  descent  method.  Note  that  if  we  fix  two  of  the  variables,  e.g.,  y  and  z, 
then  (3-3)  reduces  to  an  assignment  problem.  This  subproblem  can  be  solved  by  any  of  the  standard 
methods.  The  availability  of  these  easily  solvable  subproblems  suggests  an  algorithm  built  on  the 
Lagrangean  relaxation  technique.  In  this  case,  the  Lagrangean  block  descent  algorithm  involves 
cyclically  fixing  two  variables  at  a  time  and  optimizing  over  the  third  variable.  At  each  stage  we 
obtain  a  better  (cornerpoint)  solution  as  the  objective  function  decreases.  We  continue  iterations  of 
the  algorithm  until  there  is  no  improvement  in  the  objective  function  for  an  entire  relaxation  cycle. 
At  this  stage,  we  have  obtained  a  stationary  solution,  As  there  is  only  finite  number  of 

cornerpoint  .solutions  the  Lagrangean  relaxation  algorithm  will  terminate  finitely. 

The  number  of  major  cycles  through  the  procedure  varies  with  the  quality  of  the  initial  starting 
point.  Empirical  evidence  suggests  that  the  average  case  growth  might  be  as  low  as  logarithmic  in 
the  number  of  points,  n.  A  linear  bound  on  the  average  case  growth  seems  to  be  a  conservative 
estimate.  More  study  in  this  direction  needs  to  be  done. 

It  can  not  be  guaranteed  that  the  stationary  solution  will  be  a  global  optimum.  However,  in  the 
tests  reported  here,  the  global  minimum  was  found  by  the  Lagrangean  algorithm  every  time. 

Tree- pruning  method  of  implicit  enumeration.  After  the  initial  stationary  solution  is  found, 
we  can  effectively  use  a  tree-pruning  approach  to  verify  the  overall  optimality  of  the  solution. 
The  method  is  fashioned  after  a  common  tree-pruning  strategy  which  can  be  found  in  baird  [3], 
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zikan  and  siLBERBEBG  [15],  and  in  many  other  papers  cited  therein.  The  order  of  the  points 
px ,p7 , . . .  ,pn  is  fixed.  There  are  n2  pairs  of  points  (^,r*).  Any  of  these  pairs  can  be  matched 
to  the  point  pl .  Those  matches  that  produce  too  large  a  penalty  can  be  discarded  from  consider¬ 
ation.  No  successful  completion  of  the  triplet  selection  exists.  (An  implicit  enumeration  of  a  large 
set  of  feasible  solutions  has  been  accomplished.)  We  now  extend  one  of  the  remaining  matching 
candidates  to  another  level.  Without  less  of  generality  assume  that  the  partial  tree  consists  of  the 
triplet  (p*  ,91,r1).  At  this  stage  candidates  from  the  (n  —  l)2  remaining  pairs  are  matched  to  the 
point  p2.  For  each  candidate  the  penalty  is  added  to  the  penalty  associated  with  the  first  triplet.  If 
the  penalty  exceeds  the  threshold,  the  branch  can  be  pruned  again.  We  proceed  on  in  the  depth-first 
manner. 

The  stationary  matching  obtained  by  the  relaxation  method  is  used  as  the  initial  incumbent  in 
the  tree-pruning  branch  and  bound  method.  Examination  of  the  search  tree  of  partial  solutions 
produces  a  guaranteed  optimal  solution.  (Chances  are  that  the  Lagrangean  solution  will  be  returned 
as  optimal.)  The  analysis  of  Baird  [3]  for  the  image  registration  problem  suggests  that  the  pruning 
of  the  tree  can  be  done  in  0(n3)  average  “time” .  The  good  starting  incumbent  and  the  natural 
“steepness”  of  the  penalty  function  (that  is,  even  with  a  single  mismatched  triplet  the  associated 
penalty  is  typically  large)  together  facilitate  a  strong  fathoming  criterion.  This  conjecture  deserves 
empirical  as  well  as  analytical  study. 

In  addition  to  its  capability  to  return  the  optimal  answer,  the  value  of  the  tree-pruning  search  lies 
in  the  fact  that  all  plausible  matching  alternatives  can  be  discovered.  During  the  search  through  the 
tree,  the  fathoming  criterion  can  be  adjusted  so  that  all  assignments  with  penalty  within  of  the 
optimal  value  are  returned.  With  equal  ease  we  can  return  all  solutions  differing  from  the  optimal 
one  by  a  certain  amount,  we  can  return  the  k  best  solutions,  and  so  on. 
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4.  Computational  Complexity  of  the  Methods 


The  size  of  the  problem  is  n,  the  number  of  points  in  a  frame.  Evaluating  the  penalty  for  each  triplet 
of  points  requires  0(n3)  operations.  Therefore  an  algorithm  which  computes  the  answer  to  (3-3)  in 
approximately  0(n3)  time  is  very  efficient. 

The  computational  procedure  has  two  parts:  the  Lagrangean  section  (producing  the  initial  stationary 
point)  and  the  branch  and  bound  section  of  tree  pruning.  The  complexity  issue  of  each  section  can 
be  further  decomposed  into  smaller  units. 

Complexity  of  finding  the  stationary  point.  The  Lagrangean  part  makes  essential  use  of 
the  linear  assignment  problem.  It  takes  0(n2)  time  to  formulate  an  assignment  problem  and  0(n3) 
worst  case  time  to  solve  it.  The  average  time  required  to  solve  the  assignment  problems  is  somewhere 
between  0(n2)  and  0(n3),  KENnington  «nd  helgason  [10]. 

In  order  to  estimate  the  average  time  to  solve  the  entire  Lagrangean  problem,  we  need  to  consider 
the  expected  number  of  the  assignment  problems,  and  the  expected  time  to  solve  the  assignment 
problems  using  the  “hot”  start  technique.  At  the  moment  we  can  only  offer  a  conjecture.  It  appears 
that  the  overall  average  case  complexity  of  finding  the  stationary  point  is  0( n3)  or  better. 

If  the  above  conjecture  is  true,  we  get  the  nearly  paradoxical  result  that  the  time  to  find  a  stationary 
solution  may  be  in  the  average  sense  even  faster  than  0(n3),  the  time  to  evaluate  the  penalty  for  all 
possible  triplets.  To  explain  this  seemingly  impossible  result  we  need  to  look  back  at  the  equation 
(1-1).  The  definition  says  that  that  only  the  “inertial  portion”  of  the  penalty  involves  all  three  points. 
Therefore  if  the  distance  between  a  pair  of  points  is  quite  large,  it  becomes  possible  to  implicitly 
fathom  all  triplets  containing  the  given  pair  of  points.  The  inertia)  terms  need  not  be  evaluated 
(To  enumerate  distances  between  all  pairs  of  points  we  need  only  0(n2)  arithmetic  operations,  of 
course.) 

Tree-pruning  method.  The  time  required  to  descend  a  full  tree  branch  is  0(n3).  There  is  no 
distinction  between  the  average  case  and  between  the  worst  case  complexity. 

Although  there  are  (n!)2  leaves  in  the  tree,  experience  from  problems  with  similar  flavor  indicates 
that  all  but  a  small  number  of  branches  are  pruned  in  their  upper  levels.  A  conjecture  is  that  the 
pruning  will  also  require  0(n3)  operations  on  the  average. 
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5.  Experiment  Description 


A  Note  on  Missing  and  Spurious  Points.  All  (computet  vision)  practitioners  know  that  a 
method  that  falls  apart  when  missing  or  spurious  points  are  present  in  the  problem  is  of  academic 
importance  only.  In  the  track  initialization  problem,  missing  and  spurious  points  are  inherent.  Even 
if  there  were  no  other  technological  reasons,  we  would  still  have  the  problem  that  objects  may  either 
enter  the  field  of  view  or  exit  from  it. 

The  problem  of  detecting  (missing  and)  spurious  points  was  treated  in  zikan  [14,  Chapter  10].  In 
summary,  the  method  developed  there  was  based  on  a  combination  of  ideas  from  linear  programming 
geometry  and  from  information  theory.  By  “detection”  we  mean  pointing  out  the  most  promising 
candidate  to  be  the  spurious  point.  In  reality,  the  candidate  may  not  be  the  spurious  point,  but  it 
usually  is.  The  crude,  preliminary  version  implemented  as  a  subroutine  in  the  track  initialization 
problem  detected  the  correct  spurious  points  in  all  but  in  one  instance. 

In  our  tests  the  detected  spurious  points  were  deleted  from  consideration  for  matching.  Due  to  the 
tests’  small  problem  sizes,  the  fraction  of  spurious  points  could  become  quite  high.  At  one  time  2 
out  of  the  8  (25%)  points  were  spurious.  This  case  was  solved  successfully,  too. 

Simulated  experiment.  We  can  now  look  at  simulations  of  the  track  initialization  problem.  A  set 
of  two-dimensional  parametric  curves  was  generated  and  each  was  given  an  initial  starting  point.  The 
family  included  among  others:  line,  circle,  cardioid,  inverted  parabola,  figure  “8",  cycloid,  inverted 
square  root,  logarithmic  curve,  and  a  number  of  concatenations  of  sines  and  cosines.  Some  curves 
stayed  on  the  screen  all  time,  some  entered,  passed,  and  then  exited  from  view. 


Figure  5.1  Tracks  of  experiment’s  trajectories 
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Figure  5.1  exhibits  all  relevant  trajectories  as  they  moved  across  the  screen.  On  the  average  we  had 
8  curves  in  “sight”  at  any  given  time.f 


Figure  5.2  captures  the  situation  when  no  spurious  point  complicates  the  scene.  All  three  consecutive 
frames  are  overlaid;  the  “oldest"  points  are  marked  by  full  squares,  the  “middle  aged”  points  by 
crosses,  and  the  “youngest”  points  are  marked  by  squares  with  empty  interiors.  Figure  5. 2. a)  is 
the  picture  of  the  data  submitted  to  the  solving  subroutine.  Figure  5.1  .b)  exhibits  the  (correctly) 
estimated  trajectories.  A  natural  challenge  for  the  reader  is  to  estimate  the  solution  with  the  help 
of  the  a)  display  only. 

Figure  5.3  depicts  the  situation  of  the  same  set  of  trajectories  a  few  frames  “later”  when  two  curves 
were  concurrently  leaving  the  scene.  The  Figure  5.4  comes  from  a  still  later  time  when  a  new  curve 
entered  the  visual  field.  Note  that  the  entering  and  leaving  points  are  left  unmatched.  Otherwise 
the  discussion  from  Figure  5.2  applies,  too. 


t  The  number  may  seem  low  to  a  person  who  has  not  tried  to  do  the  track  initialization  by  hand. 
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a) 

■  frame  1,  +  frame  2,  o  frame  3 


b) 


Figure  5.2  Example  of  track  initialization  problem 
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Track  Initialization  in  the 
Multiple -Object  Tracking  Problem 

by  Karel  Zikan 

Technical  Report  SOL  88-18  Abstract 

Abstract 

LL* 

The  multiple-object  tracking  problem  involves  extraction  of  the  trajectories  ofn  moving  points  from 
(three)  successive  motion  picture  frames.  Jn  the  track  initialisation  part  of  the  problem  no  previous 
history  of  track  evolution  is  given. 

A  definition  of  a  “-three-point  metric”  functional  (analogous  to  the  classical  definition  of  distance) 
is  put  forward.  For  the  beat  estimate  of  the  trajectories,  we  partition  the  points  from  the  frames 
into  n  triplets  (baaed  on  the  three  successive  frames)  so  that  the  average  three-point  “ distance ”  is 
minimised.  The  physical  intuition  behind  this  approach  is  discussed  and  several  equivalent  math¬ 
ematical  programming  formulations  are  given.  A  practical  method  proposed  for  solving  of  the 
problem  is  based  on  a  Lagrange an  relaxation  technique,  and,  to  a  leaser  degree,  on  the  “pruning  of 
the  tree  of  “subpartitions".  On  the  basis  of  empirical  evidence  and  experience  from  related  work, 
we  conjecture  that,  on  average,  0(n3)  arithmetic  operations  are  needed  to  obtain  a  solution.  The 
problem  of  missing  and  spurious  points  in  the  images  is  also  brieBy  discussed.  A  short  summary 
and  examples  from  simulated  data  experiments  are  give n.  , 
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